Image and Video Retargetting by Darting

This paper considers the problem of altering an image by imperceptibly adding or removing pixels, for example, to fit a differently shaped frame with minimal loss of interesting content. We show how to construct a family of convex programs that suitably rearrange pixels while minimizing image artifacts and distortions. We call this ”darting” on analogy to a tailor’s dartssmall edits are discreetly distributed throughout the fabric of the image. We develop a reduction to integer dynamic programming on edit trellises, yielding fast algorithms. Oneand two-pass variants of the method have 0(1) per-pixel complexity. Of the many edits that darting supports, five are demonstrated here: image retargeting to smaller aspect ratios: adding or moving or removing scene objects while preserving image dimensions: image expansion with gaps filled by a rudimentary form of texture synthesis; temporal video summarization by ”packing” motion in time; and an extension to spatial video retargetting that avoids motion artifacts by preserving optical flow. Int. Conf. Image Analysis and Recognition This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c ©Mitsubishi Electric Research Laboratories, Inc., 2009 201 Broadway, Cambridge, Massachusetts 02139